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ABSTRACT 

The  U.S.  Air  Force  is  in  the  process  of  implementing  a  substantiaily  larger  role  for  space  operations  and  a 
new  operations  system,  the  space-based  infrared  system  (SBIRS),  to  accompany  that  role.  Despite  the 
increased  responsibility  that  will  accompany  this  new  role  and  the  implementation  of  SBIRS,  increases  in 
sateiiite  operations  personnel  may  not  occur  and  if  they  do,  they  are  unlikely  to  be  commensurate  with  the 
increase  in  responsibility.  In  this  effort  we  have  identified  SBIRS  System  Crew  Chief  (SCCH)  task 
performance  demands  that  are  likely  to  be  worsened  by  the  pending  increase  in  workload  but  \A4iich,  if 
managed  well,  can  reduce  its  negative  impact.  These  task  demands  are  event  prioritization,  task 
allocation,  and  team  communications.  In  this  paper,  we  describe  the  design  stages  and  design  of  a 
training  and  performance  support  system  called  the  Adaptive  Decision  Enabling  and  Performance 
Tracking  Toolkit  (ADEPTT)  that  will  assist  the  SCCH  manage  team  coordination  and  perform  the 
aforementioned  tasks  in  particular  during  high  workload  periods.  ADEPTT  will  be  built  using  a  cognitive 
agent  architecture  and  wili  have  four  major  components  -  supervisory  agents,  an  instructional  agent,  a 
crew  communication  tool,  and  synthetic  teammates  -  in  order  to  provide  comprehensive  training  and 
performance  support.  It  is  our  goal  to  build  ADEPTT  so  that  it  is  maximally  supportive,  minimally 
obtrusive,  has  a  minimal  learning  curve,  and  integrates  easily  into  current  training  and  operations.  In 
designing  this  toolkit,  we  followed  human-centered  design  principles,  taking  into  account  the  demands  and 
limitations  operators  already  face,  and  being  careful  to  not  add  to  existing  problems  such  as  limited 
display  space.  This  required  us  to  work  closeiy  with  members  of  the  SBIRS  operational  community  and 
make  use  of  research  tools  such  as  cognitive  task  analysis  methods  to  obtain  information  that  is  critical  to 
successful  human-centered  design  and  improved  efficiency.  It  is  our  hope  that  this  tool,  once  it  is  built 
and  implemented,  will  continue  to  be  expanded  and  refined  to  meet  the  training  and  performance  support 
needs  of  the  SBIRS  and  other  crews  as  satellite  operations  become  increasingly  complex. 
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In  response  to  current  trends  -  the  end  of 
the  Cold  War,  the  asymmetric  threat 
environment,  and  technological  advances,  in 
particular  -  the  U.S.  military  is  in  the  process 
of  modernizing  its  objectives,  priorities, 
tactics,  equipment,  and  infrastructure.  As  a 
part  of  this  modernization,  the  U.S.  Air  Force 
has  called  for  and  is  in  the  process  of 
implementing  a  substantially  larger  role  for 
space  operations.  This  larger  role  will 
encompass  significantly  greater 
responsibility.  It  will  move  space  warfighters 
off  the  sidelines  from  where  they  have,  in 
the  past,  been  limited  to  monitoring  for  the 
occurrence  of  a  limited  set  of  events.  In 
future  space  operations,  space  warfighters 
will  find  themselves  on  the  playing  field  - 
critical  participants  in  tactical  operations. 

To  succeed  in  this  new  role,  space 
warfighters  require  a  new  space  operations 
system.  This  system,  the  Space-Based 
Infrared  System  (SBIRS),  will  consist  of  two 
subsystems,  SBIRS-High  and  SBIRS-Low. 
Change  will  not  be  limited  to  the  space 
warfighter’s  equipment,  however  -  space 
personnel  will  be  required  to  undergo 
substantial  change  as  they  adapt  to  their 
new  role  and  new  equipment.  For  some 
operators,  the  implementation  of  SBIRS  will 
require  them  to  learn  very  different  ways  of 
doing  their  job,  if  not  entirely  new  jobs.  All 
operators,  however  -  regardless  of  whether 
their  basic  responsibilities  have  changed  - 
will  be  affected  by  a  dramatic  increase  in 
worldoad  associated  with  an  increase  in  the 
number  of  satellites  and  with  the  urgency  of 
supporting  tactical  operations  in  real  time. 

Steps  currently  are  being  undertaken  to 
address  the  training  and  performance 
support  requirements  that  may  accompany 
this  imminent  expansion  of  space 
operations.  In  this  particular  effort,  we  are 
looking  at  both  training  and  performance 
support  solutions  that  would  help  crew 
chiefs  and  crew  commanders  manage  their 


crews'  effectiveness  in  high  workload 
situations.  Although  the  solutions  we  design 
will  be  applicable  to  all  senior  positions  (i.e., 
crew  chiefe  and  crew  commanders),  in  this 
early  phase  of  design  we  are  focusing  on 
just  one  senior  SBIRS  position  -  the 
Systems  Crew  Chief  (SCCH). 

In  the  section  below,  we  discuss  our  general 
approach  to  assessing  the  training  and  real 
time  performance  support  needs  of  the 
SCCH.  Then  we  describe  the  SCCH 
position  and  the  crew  for  which  he/she  is 
responsible,  the  challenges  they  will  face  as 
more  and  more  SBIRS  satellites  come  on 
line,  our  proposed  training  and  performance 
support  solution  designs,  and  human- 
centered  design  considerations.  Finally,  we 
conclude  by  describing  implications  of  the 
solutions  for  contributing  to  the  effectiveness 
of  other  types  of  satellite  operations  crews. 

General  Approach 

An  important  principle  of  training  and 
support  system  design  is  to  design  with  the 
goal  of  helping  task  performers  overcome 
information  processing  limitations  that  make 
performance  vulnerable  to  error  and 
inefficiency  (e.g.,  Andriole  &  Adelson,  1995; 
Morrison,  Kelly,  Moore,  &  Hutchins  1998; 
Zachary,  1988).  For  example,  a  decision 
support  system  designed  for  crime  solving 
might  target  the  memory  recall  limitations 
and  information  processing  biases  shown  to 
affect  eyewitness  accounts  of  events  such 
as  robberies  and  automobile  accidents  (e.g., 
Loftus,  1989).  Other  limitations  that  a 
training  or  performance  support  system 
might  target  include  working  memory 
capacity,  the  speed  with  which  people  can 
perform  cognitive  operations,  attentional 
limitations,  and  multi-tasking  limitations. 
Finally,  these  systems  might  be  designed  to 
compensate  for  a  tack  of  specialized  skill 
associated  with  high-turnover  jobs,  or  jobs 
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through  which  personnel  are  cycled,  as  is 
often  the  case  in  the  military. 

In  addition  to  helping  task  performers 
overcome  challenges  such  as  these,  it  often 
is  important  that  a  support  system  support 
the  acquisition  of  expertise.  That  is,  it 
should  help  the  task  performer  understand 
domain  concepts  and  dynamics  and  support 
their  development  of  strategies  and  skills. 

Key  challenges  in  designing  training  and 
performance  support  systems  are  identifying 
the  ways  in  which  a  person's  performance 
may  be  limited  and  determining  how  a 
computer-based  technology  can  best  help 
the  person  overcome  or  circumvent  the 
identified  limitations.  These  design 
challenges  and  a  recommended  design 
approach  are  described  by  Zachary  (1988). 
Key  to  this  approach  is  access  to  subjed 
matter  experts  (SMEs)  to  both  inform  the 
system's  structure,  logic,  and  content  and  to 
assess  the  system  in  terms  of  its  usability. 

Specifically,  Zachary  recommends  the 
developer  work  with  task  performers  to  first 
define  the  highest  level  of  goal  events  -  the 
conditions  that  task  performers  are  trying  to 
achieve  during  the  course  of  performing 
their  jobs.  In  addition,  he  suggests  defining 
one  more  level  of  goals  -  the  subgoals 
beneath  those  goal  events.  These  initial 
steps  play  the  important  role  of  specifying 
both  a  focus  for  the  system  (i.e.,  supporting 
specific  goals)  and  a  framework  that  is 
consistent  with  the  way  task  performers 
approach  their  job.  Next,  Zachary 
recommends  that  the  developer  analyze 
task  performance  in  terms  of  (1)  the  context 
of  task  performance  and  associated 
demands  and  limitations  and  (2)  the 
cognitive  processes  used  to  perform  the 
task  and  cope  with  the  demands  and 
limitations.  Then,  the  developer  should 
identify  those  cognitive  processes  that 
represent  ‘weak  links',  where  the  task 
performer  may  folter  or  use  an  ineffective 
heuristic.  These  cognitive  processes 
become  the  targets  of  the  support  system. 

Once  the  targets  of  the  system  are 
identified,  the  developer  determines  which 
support  technologies  are  most  appropriate. 
In  the  present  effort,  we  considered  each  of 


the  types  of  support  listed  below  and 
identified  by  Zachary  (1988): 

•  Representation  aids  that  help  the 
operator  conceptualize  complex 
relationships,  changes  across  time,  or 
large  amounts  of  data  that  surpass 
working  memory  capacity; 

•  Process  models  that  assist  with  choosing 
among  multiple  courses  of  action; 

•  Value  models  that  help  the  operator 
assess  multiple  decision  factors  or 
attributes; 

•  Data  management  tools  that  help  the 
operator  structure  and  sort  through  all 
data  available  in  a  complex  problem; 

•  Knowledge  management  tools  that 
supplement  the  operator’s  knowledge 
base; 

•  Automated  reasoning  techniques  that 
perform  intermediate  analyses  that  the 
operator  tends  to  perform  ineffectively  as 
part  of  a  larger  decision  process;  and 

•  Decision  structuring  tools  designed  to 
support  'single-instance'  decisions. 

After  following  the  steps  above,  the  support 
system  developer  should  be  ready  to  build 
the  infomriation  content,  logic  and  outputs  of 
the  system.  This  step,  like  the  ones  before 
it,  relies  heavily  on  access  to  SMEs  and 
ideally  makes  use  of  one  or  more  cognitive 
task  analysis  techniques,  which  are  well 
suited  to  capturing  cognitive  processes, 
strategies,  knowiedge,  and  the  dynamic 
ways  in  which  these  task  elements  guide 
performance. 

Training  and  Performance  Support 
Needs 

In  order  to  identify  goals,  subgoals, 
demands,  and  limitations,  we  relied  heavily 
on  the  space  operations  community  for 
infonnation.  Demanding  work  schedules 
associated  with  bringing  a  new  operations 
system  on  line  limited  our  access  to 
personnel.  Therefore,  we  have  relied 
extensively  on  one  person  -  a  senior  space 
operations  expert  in  a  training  position  -  and 
obtained  additional  opinions  and  guidance 
from  two  fomner  members  of  space 
operations  teams.  The  following  two 
paragraphs  contain  high-level  information 
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about  the  SCCH  task,  including  information 
about  goals  and  team  context,  obtained 
while  working  with  these  SMEs. 

The  SBIRS  SCCH  oversees  and 
coordinates  the  satellite  systems  crew.  The 
satellite  systems  crew  consists  of  two 
satellite  systems  operators  (SSOs)  and 
three  ground  system  operators  (GSOs). 
Together,  the  SCCH  and  crew  have  the 
overall  responsibility  of  keeping  satellites 
healthy.  The  SCCH  and  SSOs  are 
specifically  responsible  for  tracking  and 
commanding  satellites,  monitoring  their 
health,  diagnosing  anomalies,  and 
correcting  anomalies.  The  GSOs  support 
the  SCCH,  SSOs,  and  other  SBIRS  crews 
by  establishing  and  maintaining 
communications  links  with  the  satellites  and 
other  operations  centers. 

Although  the  SCCH  is  in  control  of  satellite 
systems  operations,  the  SBIRS  Crew 
Commander  oversees  both  the  SCCH  and 
the  Mission  Crew  Chief  (MCCH).  However, 
the  MCCH  is  the  Crew  Commander’s 
primary  interest  and  the  SCCH  is  relatively 
autonomous,  at  least  in  the  current 
operations  center  configuration.  The  MCCH 
and  mission  crew  are  responsible  for 
analyzing  data  collected  via  satellite, 
influencing  the  data  collection  process,  and 
disseminating  data  to  operations  centers. 
They  interact  with  the  systems  crew 
infrequently  and  these  interactions  mainly 
are  limited  to  requests  for  the  GSOs  to  fix  or 
adjust  datalinks. 

Currently,  the  job  of  the  satellite  systems 
crew  involves  relatively  mundane  monitoring 
and  straightforward  commanding 
interspersed  with  time  pressured  anomaly 
diagnosis  and  resolution.  Scheduled 
commanding  sessions  also  can  be  time 
pressured  due  to  the  limited  window  of  time 
available.  As  the  SBIRS  system  comes  to 
be  fully  implemented,  this  situation  will 
change.  In  particular,  the  job  will  come  to  be 
much  more  characterized  by  time  pressured 
anomaly  resolution  and  commanding  and 
less  by  mundane  monitoring. 

In  addition  to  assessing  SCCH  goals  and 
task  performance  context,  our  analysis 
identified  demands  and  limitations  currently 
placed  on  the  SCCH.  These  include; 


•  A  large  information  load  -  operators 
receive  hundreds  of  pieces  of  information 
about  multiple  satellites 

•  Limited  screen  space  for  displaying  the 
large  amount  of  information  they  receive 

•  A  r^uirement  for  continuous 
monitoring/vigilance 

•  When  action  is  required,  it  typically  is  time 
critical  in  that  it  must  be  executed  at  a 
specific  time  and  within  a  limited  window 

•  Mistakes,  such  as  failure  to  detect  an 

anomaly  in  a  timely  manner  or 

transmission  of  an  incorrect  command, 
can  have  extremely  costly  consequences 

•  Many  tasks  require  coordination  among 
crew  members,  but  no  mechanisms  for 
fecilitating  that  coordination  exist;  crew 
members  who  are  not  seated  next  to 
each  other  coordinate  by  hollering  to  one 
another 

Perhaps  with  the  exception  of  the  last  item 
in  the  above  list,  system  design  features  and 
operator  strategies  have  been  developed  to 
address  SCCH  task  performance  demands 
and  limitations.  For  example,  to  help  them 
cope  with  both  the  large  amount  of 
inftsrmation  and  the  limited  display  space, 
operators  can  create  personalized 
information  displays  that  contain  only  the 
most  informative  and  diagnostic  pieces  of 
information.  This  helps  them  monitor  and 
quickly  find  the  information  they  most 
frequently  need.  As  another  example,  to 
minimize  mistakes,  procedures  have  been 
developed  whereby  no  command  is 
transmitted  to  a  satellite  until  a  second  crew 
member  has  first  checked  it.  Thus,  of  the 
demands  and  limitations  listed  above,  we 
are  focusing  only  the  last.  This  item,  which 
deals  with  team  communication,  will  be  even 
more  of  a  problem  once  SBIRS  is  fully 
implemented. 

Once  the  full  complement  of  SBIRS 
satellites  is  on  line,  many  of  the  demands 
and  limitations  identified  in  the  list  above  will 
be  exacerbated  by  the  increased  workload. 
The  SCCH  and  other  crew  members  in 
leadership  roles  will  be  faced  with 
coordinating  the  task  performance  of  crews 
that  have  to  deal  with  significantly  more  task 
demands  and  demands  that  frequently 
overlap.  Currently,  crew  leaders  do  not 
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have  strategies  or  tools  to  help  them 
coordinate  their  crews  to  deal  effectively 
with  multiple  concurrent  events.  For 
example,  they  do  not  have  strategies  or 
tools  to  help  them  prioritize  events  or 
delegate  tasks  in  a  high  workload 
environment. 

It  might  be  argued  that  additional  steps 
could  be  taken  to  improve  the  designs  and 
procedures  that  are  currently  in  place. 
However,  we  have  chosen  to  focus  on  the 
future  -  specifically,  on  facilitating  team 
coordination  under  the  anticipated  high 
workload  conditions  associated  with  the 
pending  growth  in  operations.  Tools 
designed  for  this  future  situation  will  have 
maximal  impact  because  current  operator 
strategies  and  system  design  features  for 
the  most  part  do  not  accommodate  future 
coordination  requirements.  Furthermore, 
increasing  numbers  of  satellite  operations 
teams  will  face  team  coordination 
challenges  as  advances  in  satellite  and 
communications  technologies  and  increases 
in  the  role  of  space  require  them  to  grow  in 
size  and  complexity  and  interact  with  more 
and  more  external  teams. 

Training  and  Performance  Support 
Solutions 

To  assist  the  SCCH  with  managing  and 
coordinating  the  satellite  systems  crew,  we 
are  designing  a  crew  coordination  support 
toolkit  designed  to  overcome  challenges 
associated  with  the  increased  workload 
SBIRS  operators  will  soon  experience.  The 
specific  challenges  the  tool  addresses  are 
prioritizing  events,  delegating  tasks  to  crew 
members,  and  effective  team 
communication.  This  toolkit  initially  will  be 
used  to  support  training.  However,  it  will  be 
equally  capable  of  supporting  performance 
during  real  time  operations.  This  toolkit, 
called  the  Adaptive  Decision  Enabling  and 
Performance  Tracking  Toolkit  (ADEPTT), 
will  consist  of  data  and  knowledge 
management  tools  and  automated 
reasoning  techniques.  The  specific 
components  of  the  ADEPTT  design  are; 

•  supervisory  agents  that  detect  anomalies 
and  other  events,  prioritize  them,  monitor 


SSO  and  GSO  availability,  and 
recommend  task  delegations; 

•  a  crew  communication  tool  featuring  task 
performance  shortcuts  (e.g.,  ready-to-go 
command  strings)  that  enables  the  SCCH 
to  communicate  with  the  GSOs  and  SSOs 
using  a  keyboard  and  mouse; 

•  an  instructional  agent  that  monitors 
SCCH  performance  (e.g.,  prioritization 
and  delegation)  and  that  provides  the 
SCCH  feedback  at  the  end  of  a  training 
scenario;  and 

•  synthetic  teammates  that  will  take  on  the 
role  of  GSOs  and  SSOs,  and  thereby 
make  SCCH  training  possible  in  the 
absence  of  live  team  members. 

ADEPTT  will  rely  heavily  on  cognitive  agent 
technology  -  this  technology  is  used  to  build 
and  ‘give  life  to’  the  synthetic  teammates  as 
well  as  the  supervisory  and  instructional 
agents.  The  cognitive  agent  technology  we 
are  using  is  CHI  Systems’  iGEN.  The 
iGEIM™  cognitive  agent  development 
software,  (Zachary,  Ryder,  Ross,  and 
Weiland,  1992)  provides  both  an 
architecture  for  building  synthetic 
teammates  and  agents  and  an  execution 
engine  that  allows  the  them  to  interact  in 
real  time  with  users  in  a  computer-based 
work  or  training  environment.  In  effect, 
iGEN™  is  used  to  build  a  model  of  cognitive 
task  performance  and  transform  it  into  an 
executable  agent.  IGEN™  also  provides 
various  tools  to  support  the  creation  and 
modification  of  the  cognitive  model,  as  well 
as  an  Application  Program  Interface  (API), 
which  we  call  the  agent’s  shell,  for 
communicating  with  simulation  and 
computer-based  work  environments. 

The  cognitive  architecture  and  modeling 
framework  embodied  by  iGEN™  is 
influenced  principally  by  the  work  of  Alan 
Newell  (e.g..  Card,  Moran,  &  Newell,  1983; 
Newell,  1990).  In  its  simplest  form,  Newell’s 
Unified  Theory  of  Cognition  breaks  human 
information  processing  into  three  parallel 
macro-level  mechanisms  -  perception, 
cognition,  and  motor  activity  -  and  attempts 
to  characterize  them  in  terms  of  general 
principles  derived  from  years  of  cognitive 
science  research.  Consistent  with  Newell’s 
theory,  iGEN™  includes  mechanisms  that 
represent  perception,  memory,  decision- 
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making,  and  action  and  links  these 
mechanisms  in  ways  consistent  with 
cognitive  principles  about  how  information  is 
stored,  retrieved,  and  used,  and  how  these 
processes  are  affected  by  resource 
limitations. 

This  type  of  architecture  makes  it  possible  to 
represent  human  expertise  in  a 
comprehensive  way  that  is  consistent  with 
what  we  know  about  cognition  and  how 
cognition  drives  task  performance.  Further, 
iGEN™  has  been  used  successfully  in  a 
number  of  projects  in  which  intelligent 
training  support  or  synthetic  teammates 
were  called  for  (e.g.,  Ryder,  Deaton, 
Stevens,  &  Comptois,  1998;  Zachary,  Ryder, 
&  Hicinbotham,  1998;  Zachary  et  al.,  2001). 
We  assert  that  by  using  this  type  of 
architecture,  the  ADEPTT  synthetic 
teammates  and  agents  will  be  particularly 
well-suited  to  mimicking,  tracking, 
assessing,  and  supporting  cognitive  task 
performance. 

In  the  sections  that  follow,  we  describe  each 
of  the  four  main  components  of  ADEPTT  in 
more  detail  and  then  describe  the  human- 
centered  design  principles  that  are  guiding 
the  development  of  ADEPTT. 

Supervisory  agents.  These  cognitive 
agents  will  be  built  using  data  and  findings 
obtained  during  a  cognitive  task  analysis  of 
the  SCCH  and  satellite  systems  crew 
member  jobs.  They  will  be  able  to  recognize 
a  wide  array  of  situations,  process  a  large 
number  of  situation  parameters,  make 
complex  decisions  about  event  priorities  and 
task  delegations,  determine  appropriate 
actions,  and  make  the  relevant  information, 
diagnoses/conclusions,  and  action 
sequences  available  to  the  crew  chief  or 
operator.  As  noted  above,  these  agents  will 
detect  anomalies  and  other  events,  prioritize 
them,  monitor  SSO  and  GSO  availability, 
and  recommend  task  delegations.  During 
training  exercises,  the  agents  will  serve  as 
guides  that  support  the  SCCH  in  making 
correct  decisions  or  they  may  be  deactivated 
so  that  the  SCCH  is  on  her  own.  When 
ADEPTT  is  transitioned  from  training  to 
operations,  these  agents  will  similarly  act  as 
guides,  although  their  purpose  will  be  to 
support  SCCH  performance  rather  than 
assist  with  training.  Their  outputs  will  be 


shown  in  a  single  display  window  that  can 
be  manipulated  to  utilize  minimal  display 
space. 

By  building  expert  knowledge  into  these 
agents,  they  will  be  able  to  recognize  and 
respond  to  a  majority  of  events.  However, 
just  as  humans  -  even  those  who  are 
considered  experts  in  a  domain  -  can 
encounter  an  unfamiliar  or  new  situation,  the 
ADEPTT  agents  will  not  recognize  every 
possible  situation.  Thus,  they  will  be 
programmed  to  respond  to  such  cases 
appropriately.  Specifically,  in  these  cases, 
the  agents  will  provide  the  SCCH  or 
operator  with  as  much  information  as  they 
are  able  without  making  recommendations 
that  are  contingent  on  an  accurate 
assessment  of  the  situation  (e.g.,  task 
delegation  recommendations). 

Crew  communication  tool.  Currently, 
satellite  systems  crew  members 
communicate  verbally  to  one  another,  and 
this  includes  hollering  in  order  to  convey 
infonnation  to  someone  working  in  another 
part  of  the  room.  As  the  workload  of  this 
crew  increases,  reliance  on  this  mode  of 
communicatbn  will  become  increasingly 
problematic.  The  crew  communication  tool 
will  be  designed  not  to  completely  replace 
verbal  communications,  but  to  facilitate 
communications.  The  tool  also  will  be 
critical  to  the  effectiveness  of  the 
supervisory  agent  described  above.  This 
agent  will  need  to  monitor  commands 
passed  from  the  SCCH  to  crew  members  in 
order  to  evaluate  SCCH  performance  and 
guide  SCCH  training  (although  it  is  possible 
to  build  cognitive  agents  that  recognize 
verbal  commands,  modern  speech 
recognition  technology  is  not  sufficiently 
reliable  to  make  this  an  appealing  option). 

This  tool  will  allow  the  SCCH  to  select,  via 
mouseclicks,  simple  command  strings  such 
as  ‘prepare  to  command’  and  ‘configure 
system  for  commanding’  along  with  the 
satellite  of  interest  and  send  them  to  a 
particular  crew  member  by  clicking  on 
his/her  icon.  In  addition,  it  will  allow  the 
SSOs  and  GSOs  to  send,  via  mouseclick, 
frequently  used  communication  strings,  for 
example,  to  accept  a  task  delegated  by  the 
SCCH,  to  inform  the  team  that  an  anomaly 
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is  resolved,  to  request  help,  or  to  bring  a 
new  anomaly  to  the  attention  of  the  SCCH. 

The  crew  communication  tool  will  utilize  the 
underlying  cognitive  agent  architecture  of 
ADEPT!  to  provide  situated  communication 
support.  Specifically,  supervisory  agents  will 
detect  and  evaluate  situations  and  crew 
availability.  In  the  context  of  the 
communication  tool,  the  agents  will  use  this 
information  to  offer  complete  command 
strings  to  the  SCCH  that  can  be  sent  to  an 
available  crew  member  with  a  single 
mousedick  or  keypress. 

Instructional  aoent.  The  instructional 
agent,  as  in  the  case  of  the  supervisory 
agents,  will  be  built  using  data  obtained  from 
a  cognitive  task  analysis.  This  agent  will 
have  the  same  general  types  of  capabilities 
as  the  supervisory  agents,  however  it  will 
use  those  capabilities  differently.  This  agent 
will  use  its  knowledge  about  the  SCCH  job, 
crew  activities,  and  the  constantly  changing 
environment  to  evaluate  SCCH  response 
times  and  the  accuracy  of  SCCH  decisions, 
actions,  and  communications.  At  the 
completion  of  a  training  scenario,  the 
instructional  agent  will  make  available  to  the 
SCCH  and  training  personnel  accuracy 
measures,  response  time  measures, 
process  measures  (e.g.,  did  the  SCCH 
delegate  tasks  effectively?)  and  the  overall 
outcome  of  the  situation  (e.g.,  how  much 
propellant  was  lost  before  the  leaking  tank 
was  isolated?).  In  addition,  the  agent  will 
make  available  the  performance  standards 
the  SCCH  should  endeavor  to  meet  and  will 
offer  tips  about  aspects  of  performance  on 
which  the  SCCH  should  focus  during  the 
next  training  exercise  and  about  how  to 
improve  performance. 

Synthetic  teammates.  The  SCCH  will  be 
able  to  activate  SSO  and  GSO  synthetic 
teammates  to  participate  in  training 
exercises  when  some  or  all  of  the  systems 
crew  is  unavailable.  This  capability  will 
become  increasingly  important  as  satellite 
systems  operations  become  more 
sophisticated,  increasing  the  need  for 
training,  and  as  crew  workload  increases, 
making  it  less  likely  that  a  full  set  of  crew 
members  will  be  available  at  any  given  time 
for  training.  Synthetic  teammates 
additionally  will  contribute  to  SCCH  training 


by  acting  in  ways  that  pose  realistic 
challenges  to  the  SCCH.  For  example,  they 
might  commit  anomaly  diagnosis  or 
commanding  errors,  fail  to  communicate  an 
important  piece  of  information,  not  be  able  to 
find  the  correct  command  plan  in  a  timely 
manner,  or  find  themselves  unable  to  keep 
up  with  a  large  number  of  taskings.  This 
capability  of  the  synthetic  teammates  is 
important  in  light  of  the  assertion  that 
expertise  is  gained  through  experience  with 
a  variety  of  challenging,  and  nonroutine 
events  (e.g.,  Klein,  Calderwood,  & 
MacGregor,  1989;  Salthouse,  1991;  Schmidt 
&  Bjork,  1992). 

The  ADEPT!  synthetic  teammates  will  be 
similar  to  live  teammates  in  that  their 
performance  will  be  dependent  on  their 
access  to  data  and  information  and  they  will 
change  their  behavior  appropriately  given 
detected  changes  in  the  task  environment. 
Further,  because  they  will  perform  tasks 
using  the  same  information  and  logic  as  a 
human,  they  will  be  prone  to  the  same  types 
of  performance  errors.  They  will  respond  to 
commands  recerved  from  the  SCCH  and 
task-performance  related  requests  from  one 
another  or  other  live  teammates  (e.g.,  a 
request  to  verify  a  command  before  a 
teammate  sends  it  to  a  satellite).  Similarly, 
they  will  be  able  to  send  information  such  as 
the  results  of  a  commanding  session, 
acknowledgement  of  a  command,  or 
verification  of  a  command  to  the  SCCH  and 
other  teammates. 

Design  Principles 

The  design  of  this  tool  is  guided  by  human- 
centered  design  guidelines.  In  particular, 
ADEPT!  is  designed  to  build  and  support 
expertise.  Expertise  is  characterized  by  the 
development  of  efficient  and  goal-focused 
knowledge  and  job  performance  strategies. 
Accordingly,  ADEPT!  is  designed  to  support 
efficient  goal-focused  performance  without 
interfering  with  experts’  ability  to  adapt  their 
own  personalized  strategies.  One  of  the 
design  guidelines  we  are  following  is  the 
mapping  principle,  which  holds  that  goal- 
related  data  should  be  incorporated  into 
higher-level  functional  representations  that 
are  mapped  directly  to  salient  perceptual 
properties  of  the  display  (Woods,  1991).  In 
keeping  with  this  principle,  we  are 
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highlighting  organizing  toolkit  functionality 
around  main  goals,  such  as  anomaly 
resolution. 

In  addition,  we  are  designing  ADEPT!  to  be 
flexible  -  each  display  and  function  within 
the  toolkit  will  be  accessible  from  multiple 
points  in  task  performance  and  linked  to 
multiple  displays.  For  example,  the  toolkit 
will  feature  direct  links  between  alerts  and 
response  procedures.  Through  the  use  of 
tailored  links  and  goal-focused  display 
design,  information  will  be  accessible  with 
minimal  effort.  ADEPT!  will  exhibit 
additional  flexibility  by  allowing  users  to 
disable  ftjnctions  within  the  toolkit  -  for 
example,  users  will  be  able  to  disable  the 
instructional  agent  and  synthetic  teammates, 
and  can  elect  to  not  be  shown  particular 
outputs  generated  by  the  supervisory 
agents. 

Finally,  our  design  is  influenced  by  a  prudent 
trend  toward  the  increased  use  of  critiquing 
systems  (e.g.,  Guerlain  et  al.,  1999)  to 
provide  performance  and  decision  support. 
Because  expert  systems^  do  not  produce 
optimal  responses  in  all  situations  and  can 
occasionally  steer  decision  makers  down  the 
wrong  path,  many  support  system 
developers  are  using  techniques  that  are 
supportive  of  users  while  minimally 
interfering  with  their  performance.  These 
techniques  include  improved  access  to  data, 
data  synthesis,  and  the  evaluation,  or 
critique,  of  users'  decisions  and 
performance,  accompanied  by  guidance  or 
feedback. 

Conclusions 

Despite  the  increased  operator  workload 
that  will  accompany  the  implementation  of 
SBIRS,  increases  in  satellite  operations 
personnel  may  not  occur  and  if  they  do,  they 
are  unlikely  to  be  commensurate  with  the 
increase  in  workload.  In  this  effort  we  have 
identified  SCCH  and  crew  task  performance 
demands  that  are  likely  to  be  worsened  by 
the  pending  increase  in  workload  but  which, 
if  managed  well,  can  reduce  its  negative 
impact.  These  task  demands  are  event 
prioritization,  task  allocation,  and  team 
communications.  In  the  section  above,  we 
described  a  toolkit,  ADEPT!,  which  we  have 
designed  and  will  be  building  in  order  to 


support  the  SBIRS  SCCH  with  the 
performance  of  these  tasks. 

It  is  our  goal  to  design  ADEPT!  so  that  it  is 
maximally  supportive,  minimally  obtrusive, 
has  a  minimal  learning  curve,  and  integrates 
easily  into  current  training  and  operations. 
In  designing  this  toolkit,  we  are  taking  into 
account  the  demands  and  limitations 
operators  already  face  and  are  being  careful 
to  not  add  to  existing  problems  such  as 
limited  display  space.  This  required  us  to 
work  closely  with  members  of  the  SBIRS 
operational  community  and  make  use  of 
research  tools  such  as  cognitive  task 
analysis  methods  to  obtain  information  that 
is  critical  to  successful  human-centered 
design  and  improved  efficiency. 

It  is  our  hope  that  this  tool  can  be  expanded 
and  refin^  to  meet  the  training  and 
performance  support  needs  of  the  SBIRS 
satellite  operations  crew  as  their  work 
becomes  increasingly  complex.  ADEPT! 
may  prove  useful  in  SBIRS  mission  control 
operations  as  well  -  the  mission  operations 
crew  similarly  will  experience  significantly 
increased  workload  levels  once  SBIRS  is 
fully  implemented.  In  addition,  we  anticipate 
that  portions  of  ADEPT!,  such  as  the 
situated  communications  support  or  certain 
supervisory  agents  will  be  directly  applicable 
to  satellite  crews  in  a  number  of  satellite 
control  organizations. 
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